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Abstract 



This study compared the effectiveness of ACT Composite score and high school GPA for 
predicting different levels of first-year college GPA. Logistic regression models were estimated 
for predicting first-year GPA levels ranging from 2.00 to 3.75 for a sample of postsecondary 
institutions. The prediction models estimated from one year’s data were then applied to data 
from the next year (crossvalidation over time). The resulting statistics on prediction accuracy 
were summarized across institutions. 

Both high school GPA and ACT Composite score were effective in predicting success at the 
2.00, 2.50, and 3.00 levels of first-year GPA; high school GPA was somewhat more accurate than 
ACT Composite score at these levels. High school GPA was not an effective predictor of success at 
higher levels of first-year GPA, however. For example, even a 4.00 high school GPA corresponded 
to very low probabilities of success at the 3.25, 3.50, and 3.75 levels of first-year GPA. Moreover, 
high school GPA values below 3.00 provided little differentiation among students across first-year 
GPA levels. ACT Composite score predictions, in contrast, were effective at all first-year GPA 
levels. 
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Predicting Different Levels of Academic Success in College Using 
High School GPA and ACT Composite Score 

College admissions officials typically use both high school GPA and scores on college 
entrance tests (such as the ACT Assessment) to predict, formally or informally, an applicant's 
probability of academic success in the first year of college (Breland, H., Maxey, J., Gemand, R., 
Cumming, T. and Trapani, C., 2002). Academic success is typically measured by first-year 
college GPA. 

Although high school GPA and college GPA both measure educational achievement, they 
also include other personal characteristics such as effort, attendance, conformity, and motivation 
(Goldman & Widawski, 1976; Stiggins, et al., 1989). In contrast, ACT scores primarily measure 
educational achievement in college-preparatory courses (ACT, 1997c). One might therefore 
expect high school GPA and ACT scores to be related to first-year GPA in different ways: High 
school GPA likely relates to both the cognitive and the noncognitive components of college 
GPA. ACT scores, on the other hand, likely relate only to the cognitive components of college 
GPA. 

Goldman's and others’ research in the 1970s found that the presence of non-achievement 
components in college grades was related to average student ability (e.g., Goldman & Hewitt, 
1975; Goldman, Schmidt, Hewitt, & Fisher, 1974; Goldman & Widawski, 1976). They found 
that high college grades were more likely to reflect cognitive achievement and less likely to 
reflect noncognitive factors. One might therefore expect that 

• Predictions of moderate levels of college GPA (e.g., 2.00) based on high school 
GPA are more accurate than predictions based on ACT scores, but 

• Predictions of high levels of college GPA (e.g., 3.00) based on ACT scores would 
be more accurate than predictions based on high school GPA. 
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The research comparing predictions of first-year GPA from ACT scores or high school 
GPA has largely been correlational in nature (e.g., ACT, 1998d). For example, recent summary 
statistics for institutions participating in ACT's Research Services showed that, across 129 
colleges, the median multiple correlation (across institutions) between first-year GPA and the 
four ACT scores (in English, mathematics, reading, and science reasoning) was .43 (Maxey, in 
press). The corresponding median correlation between first-year GPA and four high school 
average grades was .48. The eight-predictor ACT score/high school grade average multiple 
correlation with first-year GPA was .53. These results were not corrected for range restriction 
caused by selection on ACT scores and/or high school grades. 

Other research has examined prediction accuracy at first-year GPA levels of 2.00 or 
higher or 3.00 or higher (ACT, 1997c). This study showed that high school GPA was slightly 
more accurate that ACT scores in predicting whether students earn a 2.00 or higher GPA in the 
first year of college. For predicting whether students earn a 3.00 or higher college GPA, 
however, ACT Composite score and high school GPA had the same accuracy. The typical 
percentage of accurate predictions was 79% using either predictor, and the typical percentage of 
correct classifications using a joint ACT Composite/high school GPA model was 80%. 

The purpose of this study is to compare the accuracy of ACT Composite score and high 
school GPA for predicting successive levels of first-year college GPA. The levels considered 
were first-year GPAs of 2.00, 2.50, 3.00, 3.25, 3.50, and 3.75 or higher. 

In practice, prediction equations based on data from one freshman class are applied to the 
test scores and/or high school GPAs of future applicants. Because students may differ over time 
in their test scores, high school GPAs, or college grades, predictive validity statistics developed 
from one year's data may misstate the strength of the relationship associated with actual use of 
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such predictions. Crossvalidation allows us to study the accuracy of using prediction equations 
developed from one freshman class to forecast the success of a subsequent class. This procedure 
models the actual use of prediction equations by institutions, and it avoids the tendency of 
estimates of prediction accuracy based on a single year's data to be overly optimistic. A second 
purpose of this study, therefore, was to determine the crossvalidated accuracy of prediction 
equations and cutoffs based on ACT Composite score, high school GPA, and both variables used 
jointly. 

Data for the Study 

The data for the study were taken from two years of ACT’s Prediction Research history 
files (ACT, 1997b; 1998c). The 1996-97 file consisted of the high school GPAs (HSAV), ACT 
Composite scores (ACTC), and college grades of 219,435 first-year students from 301 
postsecondary institutions (minimum sample size = 50). High school GPAs were based on 
students' self-reports of their grades in 30 college-preparatory high school courses provided at 
the time they took the ACT Assessment. 

In addition, 728,957 nonenrolled students were identified from the 1996-97 ACT Class 
Profile history (ACT, 1997a), a database consisting of enrollment information and ACT 
Assessment records of both enrolled and nonenrolled students from over 900 institutions. 
Nonenrolled students had requested that their ACT scores be sent to at least one of the 301 
institutions, but they did not enroll at that institution. These students, plus those who actually 
enrolled in an institution and completed their first year, comprised the base-year "applicant pool" 
for that institution. In fact, some of the nonenrolled students did not apply for admission to the 
institutions to which they sent their scores, but it was not possible to distinguish the 
nonapplicants from actual applicants. The analyses in this paper are based on data from all score 
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senders; they are considered to be "applicants." All students had valid ACTC scores and HSAVs; 
enrolled students also had valid first-year college GPAs. 

The 1997-98 Prediction Research file consisted of the same elements described above for 
214,924 first-year students from 294 postsecondary institutions (minimum sample size = 50). In 
addition, 749,002 nonenrolled students were identified from the 1997-98 ACT Class Profile 
history (ACT, 1998a). These files comprised the crossvalidation-year data set. 

Operationally, first-year GPA predictions based on one year's data are applied to data for 
students two years later; for example, equations based on 1996-97 data for an institution would 
be used for applicants in fall 1998-99. However, to maximize sample sizes and the number of 
institutions for this study, the crossvalidation analysis was based on 1996-97 and 1997-98 data, 
rather than 1996-97 and 1998-99 data. In addition, Sawyer and Maxey (1980) found than 
crossvalidation statistics using a one-year period between base and crossvalidation years were 
very similar to those using a two-year period. 

The analyses were based on data from institutions that had participated in ACT’s 
Prediction Research Service in both 1996-97 and 1997-98. The resulting analysis files therefore 
consisted of data from 216 institutions: The base-year file consisted of records for 164,436 
enrolled students and 528,082 nonenrolled students, and the crossvalidation-year file consisted of 
records for 166,126 enrolled students and 539,241 nonenrolled students. 

Method 

Base-year and crossvalidation-year mean ACTC scores and mean HSAV values were 
computed by institution. Means were calculated for enrolled students, as well as for students in the 
entire applicant pool. Mean first-year GPAs were calculated by institution for students who 
completed the first year of college. Distributions of the means of these variables were then 
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summarized across institutions for both the base year and the crossvalidation year using minimum, 
median, and maximum values. 

Base-Year Logistic Regression Analysis 

Logistic regression is a method for estimating the statistical relationship between a 
dichotomous outcome variable (e.g., first-year college GPA of 2.00 or higher) and one or more 
predictor variables (e.g., ACTC score or HSAV). It follows the same general principles as linear 
regression, but fits a non-linear model with a predicted outcome bounded by 0 and 1 . Sawyer (1996) 
used logistic regression and statistical decision theory to devise a method for validating educational 
selection decisions; the method frames validity evidence in terms of probable outcomes, given the 
admissions measures and outcome criteria used. 

Validity statistics are generated from logistic regression and frequency distributions of 
scores on the admissions measure(s) to determine the effectiveness of the admissions criteria. The 
validity statistics emphasize specific cutoff values on the admissions measure and on the outcomes 
(i.e., first-year GPA). For example, the validity statistics address whether or not students with a 
specific test score would be successful in college (e.g., first-year GPA of 2.0 or higher). 

A cutoff score on a selection variable (e.g., ACTC score) is the minimum value of the 
variable for which students are selected. Admissions decisions are usually made based on multiple 
measures, some of which are subjective. ACT does not advocate making admissions decisions 
solely on the basis of a single measure, such as a test score. The use in this paper of one or two 
predictors with single cutoffs is a mathematical simplification. The methods used here, such as 
those used with the joint ACT and HSAV model, could be generalized to multiple measures, 
however. 
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We estimated three validity statistics for each predictor (or predictor combination) and 
cutoff score: 

a. the maximum percentage of correct classifications (accuracy rate (AR)), 

b. the percentage of successful students among those who would be expected to be successful 
(success rate (SR)), and 

c. the increase in the percentage of correct classifications over expecting all applicants to be 
successful (increase in accuracy rate (AAR)). 

Correct classifications include students scoring above a cutoff score who were successful and 
students scoring below the cutoff who would have not been successful, if they had been selected. 
The "optimal" cutoff score is that for which the percentage of correct classifications (AR) is highest. 

If there were no selection procedure (i.e., if all students were selected, regardless of their 
HSAVs and ACTC scores), a certain percentage of them would be successful. This percentage is 
referred to as the "baseline" accuracy rate. The arithmetic difference between the maximum 
accuracy rate and the baseline accuracy rate represents the increase in accuracy rate (AAR) that 
results from using test scores or high school GPA. 

Logistic regression models were constructed based on (a) ACTC score, (b) HSAV, and (c) 
ACTC and HSAV used jointly for predicting first-year success. The success criteria included first- 
year GPAs of 2.00, 2.50, 3.00, 3.25, 3.50, and 3.75 or higher. The logistic regression weights from 
each model were applied to the ACTC scores and/or HSAV values of all students at each institution 
with valid predictor data (i.e., the applicant pool), resulting in estimated probabilities of success for 
each student and model. 

For each institution and success criterion, optimal base-year cutoffs were identified for the 
three types of predictor models a.-c. It can be shown that optimal cutoffs also correspond to a .50 
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probability of success for a given model. For the two-predictor model, combinations of ACTC and 
HSAV cutoffs corresponding to a probability of success of .50 were identified. ARs, SRs, and 
AARs were then estimated for each predictor (or predictor combination) and optimal cutoff. All 
statistics were calculated from the conditional probabilities of each outcome for individual students 
in the applicant pool, as estimated by the regression models (Sawyer, 1996). For comparison 
purposes, median baseline accuracy rates (the percentages of all enrolled students with GPAs at or 
above each criterion level) were also reported. Distributions of these statistics were summarized 
across institutions using minimum, median, and maximum values. 

As noted previously, a probability of .50 corresponds to the maximum accuracy rate. 
Probability distributions that cross .50 will yield accuracy rate distributions that increase to a 
maximum and then decrease. If the probability distribution for an institution does not cross .50, the 
maximum accuracy rate and optimal cutoff score are generally not interpretable. Therefore, in 
reporting maximum accuracy rates for all base-year predictor models and criterion levels, the 
probability distributions for each institution were required to cross .50. 

Crossvalidation Analyses 

The accuracy of predictions based on the base-year ACTC and HSAV logistic regression 
models was assessed using the crossvalidation-year data. The logistic regression weights from each 
base-year model were applied to the ACTC scores and HSAV values of all applicants to each 
institution, resulting in estimated probabilities of success for each student and model. The base-year 
optimal cutoffs for each institution were then applied to the corresponding crossvalidation-year 
probability distributions, and crossvalidated ARs, SRs, and AARs were calculated (see p. 6 for 
descriptions of these statistics). For the two-predictor model, the logistic regression coefficients 
developed from the base year were applied to the crossvalidation-year applicant pool data to 



ERIC 



12 



8 



estimate probabilities of success. These probability "scores" were then used to calculate ARs, SRs, 
and AARs using a cutoff value of .50. Distributions of all crossvalidated statistics were summarized 
across institutions using minimum, median, and maximum values. 

Results 

Of the 216 institutions for which data were available for both 1996-97 and 1997-98, logistic 
regression models could be developed for all institutions and for all criterion values, with the 
exception of one institution for the 3.75 criterion. (For this institution, all students with a GPA of 
3.75 or higher had HSAVs of 4.00.) Different criterion levels resulted in different numbers of 
institutions for which the fitted probability curves crossed .50, however: 

• For all criterion levels except 2.00, successive increases in the criterion levels resulted in 
decreases in the numbers of institutions for which the fitted probability curves crossed .50, 
particularly for the HSAV models at the 3.25, 3.50, and 3.75 criterion levels. 

• For the 2.00 criterion level, fewer institutions had fitted probability curves based on ACTC 
scores that crossed .50. 

For the 2.50 to 3.50 criterion levels, the final sample consisted of 84 institutions with 58,482 
enrolled and 186,029 nonenrolled students for which all models and criterion levels could be 
evaluated. For the 2.00 criterion level, there were 58 institutions and 39,925 enrolled and 166,583 
nonenrolled students for which all models could be developed. For the 3.75 criterion level, there 
were only 1 5 institutions for which an HSAV model could be developed. In comparison, ACTC and 
joint models could be developed for all 84 institutions. HSAV results are therefore not reported for 
the 3.75 criterion level. Results for the HSAV, ACTC, and joint models for the 2.00 success 
criterion can be compared with each other, but they can not be directly compared to the results for 
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other criterion levels. The substantial decline in the numbers of institutions is discussed later in this 
section. 

Descriptive Statistics 

The distributions of descriptive statistics across the 84 institutions are summarized in Table 
1 . For both enrolled students and the applicant pool, median, minimum, and maximum numbers of 
students, mean ACTC scores, HSAV values, and first-year GPA (enrolled students only) are 
reported for the base year and crossvalidation year. 

TABLE 1 

Distributions, Across Institutions, of Means and Standard Deviations of 
ACT Composite Scores, High School GPAs, and First-Year GPAs, 
by Year and Applicant/Enrollment Status 



Enrollment 

status 


Predictor 

variable 


N 




Mean 


SD 


Med 


Min/Max 


Med 


Min/Max 


Med 


Min/Max 


Base year 


Applicant 

pool 


ACTC 


1,183 


219/19,675 


20.6 


17.5/26.0 


3.97 


3.35/4.81 


HSAV 


3.10 


2.76/3.65 


.59 


.37/.71 


Enrolled 

students 


ACTC 


388 


50/3,319 


21.4 


17.9/26.0 


3.79 


3.02/4.69 


HSAV 


3.18 


2.70/3.65 


.57 


. 37/.76 


First-year 

GPA 


2.63 


2.30/3.13 


.90 


.55/1.28 


Crossvalidation year 


Applicant 

pool 


ACTC 


1,268 


227/21,386 


20.9 


17.7/25.4 


4.07 


3.27/4.81 


HSAV 


3.14 


2.73/3.59 


.59 


.44Z.68 


Enrolled 

students 


ACTC 


371 


58/4,190 


21.6 


17.4/27.0 


3.84 


3.13/4.84 


HSAV 


3.22 


2.65/3.72 


.57 


.32/.71 


First-year 

GPA 


2.69 


2.05/3.13 


.88 


.48/1.48 
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As expected, the institutions' mean ACTC scores and HSAVs were typically higher 
among enrolled students than among the students in the entire applicant pool. The corresponding 
standard deviations were smaller for enrolled students. The distributions of crossvalidation year 
mean ACTC score, HSAV, and first-year GPA showed slightly higher values than those for the 
base year. 

Mean ACTC scores for enrolled students from both years were typically lower than those 
for first-year college students nationally (mean ACTC = 21.7; ACT, 1998e). Mean HSAVs were 
similar to those for first-year college students nationally (mean HSAV = 3.23; ACT, 1998e). 
Base-Year Logistic Regression Results 

Figures 1 and 2 show the median probabilities corresponding to all six criterion levels for 
ACTC (Figure 1) and HSAV (Figure 2) models. The probabilities were summarized across the 
216 institutions (215 institutions for the 3.75 criterion level) for which all three models could be 
developed. 

As shown in Figure 1 , the median probability distributions for all criterion levels ranged 
from near zero for an ACTC score of 1 to between .83 and .98 for an ACTC score of 36. A 
student with an ACTC score of 21 (the approximate median mean ACTC score across the 84 
institutions) would typically have a .81 probability of earning a 2.00 first-year GPA or higher. 
The corresponding probabilities for the other criterion levels would be .62 (2.50), .36 (3.00), .20 
(3.25), .11 (3.50), and .04 (3.75), respectively. 
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FIGURE 1. Median Probabilities of 2.00, 2.50, 3.00, 3.25, 3.50, and 3.75 or Higher First- 

Year GPA, Based on ACT Composite Score 




As shown in Figure 2, the median probability distributions for HSAV ranged from near 0 
(HSAV = 0), to between .29 and .93 (HSAV = 4.00). A student with an HSAV of 2.00 would 
typically have a .48 probability of a 2.00 or higher first-year GPA, and a .21 probability of a 2.5 
or higher first-year GPA. The corresponding median probabilities for the other criterion levels 
would be .05 or lower. In comparison, a student with an HSAV of 3.2 (the approximate median 
mean HSAV across the 84 institutions) would typically have a .83 probability of a 2.00 or higher 
first-year GPA. The corresponding median probabilities for the other criterion levels would be 
.64 (2.50), .37(3.00), .19 (3.25), .11 (3.50), and .03 (3.75), respectively. 

Note that for the criterion levels of 3.50 and 3.75, an HSAV of 4.00 corresponded to a 
median probability of success of less than .50. Moreover, for the criterion levels of 2.50 and 
3.00, there was little difference in the median probabilities for HSAV values of less than 2.00. 
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Similarly, for HSAV values of 2.50 for the 3.00, 3.25, 3.50, and 3.75 criterion levels, there was 
little difference in the corresponding median probabilities. Any substantive differentiation among 
students’ probabilities across all criterion levels therefore appeared to occur between HSAV 
values of 3.0 and 4.0. 



FIGURE 2. Median Probabilities of 2.00, 2.50, 3.00, 3.25, 3.50, and 3.75 or Higher First- 
Year GPA, Based on High School GPA (HSAV) 




Loss of Institutions Due to Lack of Model Fit 

A probability curve for a predictor variable is required to cross .50 for there to be a 
maximum accuracy rate. For example, the typical probability of a 3.25 or higher first-year GPA 
for a student with a 4.00 HSAV was less than .50. We refer to this type of model as a "nonviable 
model" for an institution. Models for institutions with probability curves crossing .50 are referred 
to as “viable models.” Table 2 includes the numbers of viable and nonviable models, by 
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predictor. Note, for the 3.75 criterion level at one institution, all students with GPAs of 3.75 or 
higher had HSAVs of 4.00. 



TABLE 2 

Number of Institutions Where Probability Distributions Did and Did Not Cross .50 



Success 

criterion 

level 


Predictor 

variable 


Number of instil 


tutions 


Viable 

models 


Nonviable 

models 


Total possible 
models 


2.00 or 
higher 


ACTC 


151 


65 


216 


HSAV 


207 


9 


216 


ACTC+HSAV 


210 


6 


216 


2.50 or 
higher 


ACTC 


214 


2 


216 


HSAV 


216 


0 


216 


ACTC+HSAV 


216 


0 


216 


3.00 or 
higher 


ACTC 


211 


5 


216 


HSAV 


209 


7 


216 


ACTC+HSAV 


214 


2 


216 


3.25 or 
higher 


ACTC 


208 


8 


216 


HSAV 


172 


44 


216 


ACTC+HSAV 


210 


6 


216 


3.50 or 
higher 


ACTC 


200 


16 


216 


HSAV 


98 


118 


216 


ACTC+HSAV 


203 


13 


216 


3.75 or 
higher 


ACTC 


168 


47 


215 


HSAV 


15 


200 


215 


ACTC+HSAV 


175 


40 


215 



As can be seen in Table 2, there was a large number of nonviable models for HSAV, 
especially for the 3.25, 3.50, and 3.75 success criterion levels. For these criterion levels, 20%, 
55%, and 93%, respectively, of the HSAV models were nonviable, compared to 4%, 7%, and 
22%, respectively, of the ACTC models. In general, institutions with nonviable ACTC models 
also had nonviable HSAV models. In contrast, for the 2.00 criterion level, 30% of the ACTC 
models were nonviable, compared to 4% of the HSAV models. For all of the nonviable ACT 
models for this criterion level, all fitted probabilities of success exceeded .50. 



ERIC 



18 



14 



The institutions with nonviable models were investigated further to attempt to explain the 
large discrepancy between the ACTC and HSAV results. The criterion level of 3.50 was 
targeted, because of the relatively large numbers of nonviable models for HSAV. The data file 
was matched to ACT’s Institutional Data Questionnaire history (ACT, 1998b) to obtain 
descriptive information about these institutions. Of the 216 institutions possible, the 102 with 
nonviable models represented all regions of the country, both two-year and four-year institutions 
(29% two-year, 69% four-year, and 3% unable to determine), and all types of admissions policy 
(2% highly selective, 16% selective, 27% traditional, 13% liberal, 41% open, and 2% unable to 
determine). When compared with the institutions with viable models, very few differences were 
found in their ACTC scores, HSAV, or first-year college GPA distributions. 

Validity Statistics 

Table 3 shows median baseline accuracy rates, optimal cutoff scores, estimated accuracy 
rates (ARs), estimated increases in accuracy rates (AARs), and estimated success rates (SRs) for 
the 84 institutions for which validity statistics could be calculated. 

As one would expect, median optimal ACTC and HSAV cutoffs increased across 
criterion levels from 2.00 to 3.75. For example, the median optimal ACTC score for a GPA level 
of 2.50 or higher was 18; the corresponding optimal cutoff scores for the other criterion levels 
were 22, 25, 27, and 30, respectively. Note, however, that statistics could not be calculated for 
the HSAV model for the 3.75 criterion level, due to the substantial numbers of institutions where 
the probability of a 3.75 or higher GPA for students with a 4.00 HSAV was less than .50. (The 
probability distributions did not cross .50 for 69 of the 84 institutions.) 
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Correspondingly, median baseline accuracy rates (median percentages of students with 
GPAs at or above each criterion level) decreased across all criterion levels. A relatively high 
percentage of students (median = 75%) had GPAs of 2.00 or higher. 

TABLE 3 

Medians, Across 84 Institutions, of Base-Year Logistic Regression Statistics 



Success 

criterion 

level 


Baseline 

accuracy 

rate 


Predictor variable 


Optimal 

cutoff 


Accuracy 

rate 

(AR) 


Increase in 
accuracy rate 
(AAR) 


Success 

rate 

(SR) 


2.00 or 

higher 

GPA (I) 


75 


ACTC 


14 


.76 


.00 


.77 


HSAV 


2.21 


.79 


.02 


.80 


ACTC & HSAV (3) 




.79 


.02 


.81 


2.50 or 

higher 

GPA 


61 


ACTC 


18 


.69 


.07 


.70 


HSAV 


2.78 


.71 


.09 


.73 


ACTC & HSAV (3) 




.74 


.11 


.75 


3.00 or 

higher 

GPA 


39 


ACTC 


22 


.71 


.31 


.65 


HSAV 


3.39 


.73 


.32 


.67 


ACTC & HSAV (3) 




.76 


.36 


.70 


3.25 or 

higher 

GPA 


25 


ACTC 


25 


.79 


.54 


.63 


HSAV 


3.73 


.79 


.52 


.60 


ACTC & HSAV (3) 




.81 


.57 


.67 


3.50 or 

higher 

GPA 


17 


ACTC 


27 


.84 


.67 


.61 


HSAV 


3.91 


.83 


.65 


.55 


ACTC & HS AV (J) 




.86 


.69 


.64 


3.75 or 
higher 

gpa (2) 


7 


ACTC 


30 


.93 


.85 


.57 


Rhsav 


v ! .d;Vf7V 


. r J- :■ '%'? ■■ X 


BuM 'll 




ACTC & HSAV (3) 


;T-:' 7 ’.H 


.93 


.86 


.59 



Notes: 

(1) All 2.00 models are based on 58 institutions for which the base-year fitted probability distributions 
based on ACTC scores crossed .50. 

(2) HSAV prediction statistics could be calculated for only 15 of the 84 institutions 

(3) A range of optimal combinations of ACTC score and HSAV correspond to a probability of 
.50 for the joint model. 



For criterion levels of 2.00, 2.50, and 3.00, the median ARs and AARs indicated that the 
HSAV models were somewhat more accurate than the ACTC models. However, for criterion 
levels of 3.25 and 3.50, the median ARs for the ACTC equaled and then exceeded those for 
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HSAV. (The corresponding median AARs were both somewhat higher for the ACTC models.) 
For the 3.75 model, the median ARs for the ACTC model and the joint model were identical, 
reflecting the small contribution of HSAV to the joint model. For all criterion levels except the 
2.00 level, the median ARs and AARs for the ACTC and HSAV joint model exceeded those for 
the single-predictor ACTC and HSAV models. 

Median SRs showed a similar result: For criterion levels of 2.00 and 2.50, median SRs 
for the HSAV model were higher than those for the ACTC model. For all other criterion levels, 
the median SRs for ACTC were higher than those for HSAV. For all criterion levels, median SRs 
for the joint model exceeded those for the separate HSAV and ACTC models. 
Crossvalidation-Year Logistic Regression Results 

The crossvalidated logistic regression results are shown in Table 4. Median baseline 
accuracy rates and crossvalidated estimated accuracy rates (ARs), increases in accuracy rates 
(AARs), and success rates (SRs) are shown for each criterion level. The median optimal cutoffs 
for the ACTC, HSAV, and joint models were the same as those for the base-year. For all 
criterion levels except 2.50 or higher, median baseline accuracy rates were slightly higher for the 
crossvalidation year than for the base year. 

The crossvalidated ARs and AARs were very similar to those for the base year: 
Differences between base-year and crossvalidated median AR did not exceed .02 for all three 
models. Differences between base-year and crossvalidated median AARs for all three models 
were .04 or less across all criterion levels. 

The base-year results showed somewhat greater prediction accuracy for the HSAV model 
for criterion levels of 2.50 and 3.00, similar prediction accuracy at a criterion level of 3.25, and 
somewhat greater prediction accuracy for the ACTC model at a criterion level of 3.50. The 
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crossvalidated results differed slightly: The crossvalidated median ARs for the 3.25 level slightly 
favored the HSAV model (.77 vs. .78), although the median AARs were the same (.50) for the 
two models. Crossvalidated ARs and AARs continued to favor the ACTC model for the 3.50 
criterion level. For the 3.75 level, the crossvalidated median AARs for the ACTC and joint 
models were the same and the corresponding median ARs differed by .01 (.92 and .93, 
respectively), again illustrating the weak contribution of HSAV to the joint model for this 
success criterion. 



TABLE 4 

Medians, Across 84 Institutions, of Crossvalidated Logistic Regression Statistics 



Success 

criterion 

level 


Baseline 
accuracy rate 


Predictor variable 


Accuracy 

rate 

(AR) 


Increase in 
accuracy rate 
(AAR) 


Success 

rate 

(SR) 


2.00 or 

higher 

GPA (1) 


78 


ACTC 


.78 


.00 


.80 


HSAV 


.79 


.01 


.83 


ACTC & HSAV 


.80 


.02 


.83 


2.50 or 

higher 

GPA 


61 


ACTC 


.69 


.06 


.70 


HSAV 


.72 


.08 


.74 


ACTC & HSAV 


.73 


.10 


.76 


3.00 or 

higher 

GPA 


40 


ACTC 


.70 


.29 


.63 


HSAV 


.73 


.31 


.66 


ACTC & HSAV 


.75 


.35 


.70 


3.25 or 

higher 

GPA 


26 


ACTC 


.77 


.50 


.61 


HSAV 


.78 


.50 


.61 


ACTC & HSAV 


.81 


.53 


.68 


3.50 or 

higher 

GPA 


19 


ACTC 


.83 


.65 


.58 


HSAV 


.82 


.64 


.54 


ACTC & HSAV 


.86 


.67 


.67 


3.75 or 
higher 

gpa (2) 


8 


ACTC 


.92 


.85 


.53 


HSAV 






’ V'l.’ • '7 \ "■ ' ' ; 


ACTC & HSAV 


.93 


.85 


.71 



Notes: 

(l) All 2.00 models are based on 58 institutions for which the base-year fitted probability distributions 
based on ACTC scores crossed .50. 

<2) HSAV prediction statistics could be calculated for only 1 5 of the 84 institutions. 
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Crossvalidated median SRs were slightly higher than were those for the base year for 
lower criterion levels (2.00 and 2.50). For higher criterion levels, median SRs were lower than 
were those for the base year, with the exception of the 3.75 criterion level. For this level, the 
median SR for the joint model was considerably higher for the crossvalidation year than for the 
base year. This result might be attributable to the higher baseline accuracy rates for the 
crossvalidation year sample. 

Conclusions 

An important finding of this study is the apparent inability of HSAV to predict high 
levels of academic achievement during the first year of college. For 20%, 55%, and 93% of the 
institutions, a 4.00 high school GPA corresponded to a probability of a 3.25, 3.50, and 3.75 or 
higher GPA that was less than .50. Moreover, in some cases HSAV values less than 3.00 
provided little discrimination in terms of student's chances of achieving different first-year 
GPAs. This evidence suggests that noncognitive factors contribute significantly to high school 
grades lower than a B. 

Consistent with prior research (ACT, 1997c), HSAV predictions of first-year GPAs of 
2.50 and 3.00 were somewhat more accurate than those based on ACTC score. However, 
predictions based on ACTC score and HSAV jointly were more accurate than those based on 
HSAV or ACTC score alone. For higher first-year GPA criterion levels, ACTC score predictions 
were as accurate or more accurate than those associated with HSAV. These findings appear to 
support the conclusions of earlier research that college grades reflect achievement and 
noncognitive factors (Goldman, et al., 1974; 1975; 1976; Beck, 1999; Pothoven, 1993; Lambert, 
1993; McSpirit, Kopacz, Jones, & Chapman, 2000; U.S. Department of Education, 1994). The 
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findings also support earlier research that these noncognitive factors are less pervasive at higher 
achievement levels (Goldman et al., 1974; 1975; 1976). 

Postsecondary institutions seek high achievement for their students, and want to admit 
students who have a good chance of being successful in college. Some people might criticize the 
use of standardized admissions tests, stating that such tests are appropriate only for “elite” 
students, and should not be used for the typical college-bound student. These results suggest that 
ACT Composite scores provide greater differentiation across levels of achievement than do high 
school GPAs in terms of students’ probable success during their first year in college. 
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